Skip to content

docs: add ALP and fastalp floating-point compression to concept design - #364

Open
x-at-01 wants to merge 1 commit into
cnosdb:mainfrom
x-at-01:feat/add-fastalp-compress
Open

docs: add ALP and fastalp floating-point compression to concept design#364
x-at-01 wants to merge 1 commit into
cnosdb:mainfrom
x-at-01:feat/add-fastalp-compress

Conversation

@x-at-01

@x-at-01 x-at-01 commented Sep 4, 2026

Copy link
Copy Markdown

Motivation & Fit

In CnosDB concept design documentation (docs/reference/concept_design/compress.md), Gorilla and Pco are presented as the primary floating-point compression algorithms. While Gorilla is efficient for temporal XOR patterns and Pco provides high compression via tANS entropy coding, decimal-scaled time-series floats (such as sensor measurements, telemetry, market prices, and coordinates) benefit substantially from adaptive decimal extraction.

I would like to propose adding an introduction to ALP (Adaptive Lossless floating-Point Compression) and its pure Rust implementation fastalp (available on crates.io) to both Chinese and English concept documentation.

Summary of Changes

  • docs/reference/concept_design/compress.md: Added ALP / fastalp section detailing its working principle (adaptive sampling, 10^e decimal scaling, Frame-of-Reference/Delta-ALP, vectorized bitpacking, and lossless exception handling) and applicability comparison with Gorilla and Pco.
  • i18n/en/docusaurus-plugin-content-docs/current/reference/concept_design/compress.md: Added the corresponding English documentation.

Key Architectural Highlights

  • Zero heap allocation: streaming compress_into and decompress_into interfaces.
  • SIMD bitpacking: vectorized bit-unpacking routines delivering high-throughput scans.
  • Adaptive dual-mode encoding: automatic selection between FOR and Delta-ALP for stationary vs monotonic series.
  • Lossless guarantees: 100% bit-exact roundtrips with dedicated exception storage for non-decimal outliers.

Measured Results

On standard benchmarks and real-world time-series datasets:

Algorithm Compression Ratio Compression Speed Decompression Speed
Gorilla ~1.5x - 2.0x 1.2 GB/s 1.4 GB/s
Pco ~3.0x - 4.5x 0.22 GB/s 0.35 GB/s
fastalp ~2.5x - 3.5x 4.69 - 6.68 GB/s 31.17 GB/s

Testing

  • Verified markdown formatting and link integrity across Chinese and English documentation files.

fastalp benchmark

@netlify

netlify Bot commented Sep 4, 2026

Copy link
Copy Markdown

Deploy Preview for docs-cnosdb-deploy ready!

Name Link
🔨 Latest commit 90dd11f
🔍 Latest deploy log https://app.netlify.com/projects/docs-cnosdb-deploy/deploys/6a9a4bb158b94f000860b844
😎 Deploy Preview https://deploy-preview-364--docs-cnosdb-deploy.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@x-at-01 x-at-01 closed this Sep 4, 2026
@x-at-01 x-at-01 reopened this Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants